Nonsmooth optimal value and policy functions for mechanical systems subject to unilateral constraints

State–of–the–art approaches to optimal control of contact–rich robot dynamics use smooth approximations of value and policy functions and gradient–based algorithms for improving approximator parameters. Unfortunately, the dynamics of mechanical systems subject to unilateral constraints—i.e. robot locomotion and manipulation—are generally nonsmooth. We show that value and policy functions generally inherit regularity properties like (non)smoothness from the underlying system’s dynamics, and demonstrate this effect in a simple mechanical system. We conclude with a discussion of implications for the use of gradient–based algorithms for optimal control of contact–rich robot dynamics.